1. Field of the Invention
The present invention relates to an image processing apparatus for converting moving image data having a first frame rate into moving image data having a higher frame rate, and a method of controlling the same.
2. Description of the Related Art
Conventionally, a CRT has been synonymous with a moving image display device for, for example, television. However, so-called liquid crystal displays, plasma displays, and FED displays have been put into practical use in recent years. That is, there are now displays of various types.
The displays of these types adopt different display methods. For example, display devices based on a liquid crystal device (e.g., direct-view-type liquid crystal display device, liquid crystal rear projector, and liquid crystal front projector) use many scanning methods. In any case, the light output period in each pixel portion occupies a large part of the display period of one frame. For this reason, such a display device is called a hold-type display device.
On the other hand, in, for example, a CRT or FED, light is output in each pixel portion once in a frame. The light emission time is much shorter than the frame display period and is normally 2 msec or less. For this reason, such a display device is called an impulse-type display device.
There also exist a so-called plasma display and a field sequential display which are of types different from the above-described classes.
The display methods of the respective types have the following features.
(1) Hold-Type Display Device
A display device of this type emits light during a large part of a frame period. Hence, the temporal imbalance of light intensity is small, and flicker is rarely observed. Additionally, pursuit (i.e., the pursuit by the eyes of a moving portion of a moving image) makes motion blur relatively large in accordance with the length of the light emission period in a frame. “Motion blur” here is different from that caused by the response characteristic of a display device.
(2) Impulse-Type Display Device
A display device of this type emits light in a very short time during a frame period. Hence, the temporal imbalance of light intensity is large, and flicker synchronous with a frame is observed. However, motion blur in pursuit is rarely observed. It is therefore possible to obtain a resolution almost equal to that of a still portion.
In general, the light emission period of a display device changes depending on the display method and display device. The above-described types (1) and (2) are diametrically opposed in terms of the light emission period. The longer the light emission period (corresponding to the hold time) in each method is, the larger the motion blur in pursuit is. The shorter the light emission period is, the smaller the motion blur is. That is, the light emission period and the magnitude of motion blur are almost proportional to each other. On the other hand, concerning flicker synchronous with a frame, the longer the light emission period is, the smaller the flicker observed. The shorter the light emission period is, the larger the observed flicker. That is, the light emission period and flicker have a trade-off relationship.
A solution to the two problems is multiplying the frame frequency by N. In many case, N=2. That is, the rate is doubled. When the frame frequency is doubled, the light emission period in each double-rate frame is halved. This also almost halves the motion blur. Regarding flicker as well, if an initial frame frequency of 60 Hz is doubled to 120 Hz, the frequency of flicker falls outside the response characteristic of human eyes. Hence, no flicker is observed.
As described above, doubling the frame frequency (more generally speaking, multiplying the frame frequency by N) has a large effect but poses a new problem.
For example, when the frame frequency of an original image signal is 60 Hz, the image information is updated every 1/60 sec. If the frame frequency is doubled to display image data at 120 Hz, necessary image information is missing every other frame. As a measure, identical images are displayed, for example, twice if the frame frequency is doubled. This solves flicker but cannot improve motion blur in the original image. In an impulse-type display device, doubled images are observed in pursuit (this phenomenon will be referred to as “double-blurring” hereinafter).
To reduce the motion blur or double-blurring and prevent flicker, two methods are mainly used to double the frame frequency.
The first method detects the motion of an object in an original image and estimates images between two frames. This is generally called an intermediate image generation method by motion compensation. In this first method, an estimation error occurs under a specific condition. In addition, the amount of computation required is extremely high.
In the second method, a filter process is first performed for each frame of an input image to separate a spatial high-frequency component strongly related to motion blur and a spatial low-frequency component strongly related to flicker. The spatial high-frequency component is concentrated in one sub-frame (one of the two double-rate frames corresponding to the original frame). The spatial low-frequency component is distributed to both sub-frames (both of the two double-rate frames corresponding to the original frame).
In this specification, this second method will be called a “method of separating an image into spatial frequencies and distributing them to sub-frames for display”.
As the “method of separating an image into spatial frequencies and distributing them to sub-frames for display”, Japanese Patent Laid-Open No. 6-70288 (to be referred to as patent reference 1 hereinafter), Japanese Patent Laid-Open No. 2002-351382 (to be referred to as patent reference 2 hereinafter), and U.S. Pre-Grant Publication No. 2006/0227249A1 (to be referred to as patent reference 3 hereinafter) are known.